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A Study of the Effectiveness of Teacher Appraisal 
in Hong Kong Self-Managing Schools 

Kim Wcm Mo 

Yuen Long Lutheran Secondary School 
Tin Yiu Estate, Tin Shui Wai, Yuen Long, NT, Hong Kong 



Background to the study 

Introduction 

There have been numerous educational reforms in the western world (Beare and Boyd, 
1993; Gamage, 1996), the purpose of which is to improve the outcomes of students through 
effective teaching and learning (Caldwell and Hayward, 1998). Matters like school goals, 
governance, finance, curriculum, and student assessment have all received attention. However, 
teachers and the quality of their instruction have received more scrutiny than any other areas 
(Duke, 1995). The recent emphases of organizational productivity, performance and individual 
accountability have made performance appraisal of personnel an essential process in contemporary 
organizations (Castetter, 1992). In the education sector, effective evaluation of educational 
personnel has been found to be increasingly important in improving schools (Lane, 1990) and in 
contributing to a self-managing school's self-review process (McConnell, 1 995). 

The Inspection of Teachers 

In Hong Kong, the Advisory Inspectorate Division of the Education Department is 
responsible for the supervision and appraisal of school teachers. The Inspectorate carries out 
inspection visits to schools, the major purposes of which are to monitor and improve the quality 
of teaching (Education Department, 1992a). During an inspection, the inspector observes lessons 
and inspects students' exercise books. After classroom observation, the inspector provides oral 
feedback to the teacher on matters related to teaching skills and classroom management. 
However, the school does not receive any formal, written report about the performance of the 
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teachers being inspected. After the inspection, the inspectors cannot provide follow-up 
supervision or organize staff development programs for the teachers because they are not 
members of the school. On the other hand, the school does not have any information from 
inspectors about teachers' needs and deficiencies to plan staff development activities for them. 
Also, because of the large number of schools, teachers cannot be inspected regularly (Education 
Commission, 1994). It can be seen that apart from serving mainly accountability purposes, both 
the school and the teachers benefit little from such occasional inspections. Apart from what has 
been described, teacher appraisal has not been commonly practiced in Hong Kong schools 
(Cheng, 1992) as it is not required by the Education Department. 

Consequences of the Educational Reforms in Hong Kong 

Based on the findings of effective schools research and responding to the worldwide school- 
based management movement, the Education and Manpower Branch and the Education 
Department jointly published a report in March 1991 called The School Management Initiative: 
Setting the Framework for Quality in Hong Kong Schools. The report proposed a model of self- 
managing schools, like the one described by Caldwell and Spinks (1988), in which schools were 
to be given greater flexibility in financial and personnel management. In the report, it was pointed 
out that it is necessary to have formal procedures for evaluating the performance of teachers so 
that management decisions can be made based on staff strengths and weaknesses, and schools may 
not be accused of unfairness in staff promotion. In 1994, the Education Commission proposed a 
concept of quality assurance for achieving quality education in schools and regarded teacher 
appraisal as one of the internal quality assurance mechanisms. In 1997, the Education Commission 
has outlined a quality assurance framework and recommended all schools to implement teacher 
appraisal before 1998, two years before they were required to have put in place school-based 
management. 



Purposes of Appraisal 

Millman (1981) and Wise, Darling-Hammond, McLaughlin and Bernstein (1985) have noted 
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that teacher appraisal systems can be classified as formative or summative, each serving its own 
purposes. Andrews and Barnes (1990) considered formative appraisal as process evaluation and 
summative appraisal as product evaluation. Process evaluation provides information for teachers 
so that they can make adjustments during the appraisal period. With product evaluation, a value 
is placed on the performance of teachers and that value can be used in making decisions about 
contract renewal, assignment to a rung on a career ladder or an award of incentive pay. For 
Stiggins (1986), the purpose of formative appraisal is to provide information on teachers' 
strengths and weaknesses in order to plan for remedial training, thus, promoting their professional 
development, and therefore he has argued that formative systems are the best for teachers and 
schools. Summative appraisal, on the other hand, provides information for personnel management 
decisions and promotes educational accountability. The concern for accountability for improved 
student learning has led to the development of summative systems (Schalock and Schalock, 1993; 
Wright, Horn and Sanders, 1997). Both systems of appraisal are important as they both aim at 
improvement of schools by improving teachers and by making sound personnel decisions 
(Bridges, 1990; Duke and Stiggins, 1990). Therefore, some schools choose to implement both 
systems separately while others implement either one or the other (Valentine, 1992). 

According to the Education Department (1992b), teacher appraisal in Hong Kong schools 
participating in the SMI scheme is expected to serve both formative and summative purposes, 
including helping teachers improve their performances through appropriate staff development and 
providing information for making personnel decisions such as promotion and disciplinary actions. 



Teacher Appraisal 

Larson, (1984) has noted that a well-planned and carefully implemented teacher appraisal 
system can have a far-reaching impact on teacher effectiveness, while a poorly-planned one can 
dampen staff morale and have a negative effect on teacher performance. Kimball (1980) also 
noted that sound appraisal practices not only enable personal and professional growth of teachers 
but also form the bases for fair and constructive personnel decisions. What are the aspects of an 
appraisal system that are considered important in determining its effectiveness? 
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If an appraisal system does not have a clear purpose, it will just be a meaningless exercise 
(Stronge, 1991). For an appraisal to be effective in bringing about teacher improvement, it is 
important to emphasize the formative purpose of appraisal (Root and Overly, 1990). It is also 
very important that these purposes be understood (Valentine, 1992) and perceived by teachers 
as helping them improve (Darling-Hammond, Wise and Pease, 1983; McGreal, 1982). 

Classroom observation is an important component of the appraisal process (Evertson and 
Burry, 1989; Stodolsky, 1990). It usually starts with a planning conference, followed by 
information gathering through actual observation, and concludes with a post-observation 
conference (Bollington, Hopkins and West, 1990; Poster and Poster, 1993). These are also the 
major procedures of clinical supervision (Goldhammer, Anderson and Krajewski, 1993) which are 
typically formative in nature (Sergiovanni and Starratt, 1993). For appraisal systems which also 
serve formative purposes, summative appraisal marks the end of an appraisal cycle. The appraisal 
data collected in the formative phase form the basis for writing up a summative appraisal report 
(Valentine, 1992). 

In gathering data for appraisal, it is important to note that teaching and learning are complex 
acts that occur in many forms and contexts. Thus, it is necessary to collect information about 
classroom performance from a wide range of sources (McGreal, 1983; Murphy, 1987). Recently, 
there are growing emphases on using student achievement data in appraising teachers as they 
reflect the effectiveness of teaching (Schalock and Schalock, 1993; Wright, Horn and Sanders, 
1997). 

The criteria forjudging the performance of a teacher depends on the purpose of appraisal 
(Castetter, 1992). If the purpose of appraisal is teacher improvement, teachers' need and levels 
of competence at different stages of their career should be considered ( Glattom and Holler, 
1987). Moreover, it is inappropriate to rank them by comparison with the performances of other 
teachers for it will invariably promote defensiveness toward appraisal (Stiggins and Bridgeford, 
1985). 

Appraisers are important as they have to make judgments and offer suggestions to the 
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teachers. Therefore, their relationship with the appraisee is crucial to successful outcomes of 
appraisal (Duckett, 1991). If teachers are appraised by someone who is at a higher rank in the 
school, it is important that the appraiser be credible, respected and skillful in appraising teachers 
so as to eliminate the fear of misuse of appraisal data (McNamara, 1995). For suggestions to be 
accepted by the teachers and to facilitate growth, the appraisers should be perceived as helpful, 
patient, trustworthy and credible in providing useful information, able to demonstrate new ideas 
and techniques, and to persuade teachers with convincing reasons (Duke and Stiggins, 1986). 

Valentine (1992) has noted that implementing a teacher appraisal system inevitably results 
in changes to the existing conditions. He therefore suggested that for changes to be lasting and 
appraisal to be effective, those affected should be involved in developing, refining, and 
implementing the system. Through involvement in the appraisal process, teachers can also reflect 
their interests, concerns, aspirations, and needs (Stiggins and Bridgeford, 1985). 

The most important purpose of appraisal is the improvement of teachers’ performances. 
This can be better achieved if appraisers provide feedback on the problems identified (Natriello, 
1990a). Indeed, the impacts of appraisal on teachers depend on the nature and quality of the 
feedback provided to the teachers (Machell, 1995; Stiggins and Duke, 1988). Feedback is 
effective when it is immediate, direct (Chimside, 1984), timely, specific, nonpunitive (McLaughlin 
and Pfeifer, 1988) and provides suggestions for improvement (Duke and Stiggins, 1986). To 
make the suggestions convincing and acceptable, it is important to develop a rationale for each 
comment made (Goodman, 1988). 



Purpose of Study 

Formal teacher appraisal was introduced to Hong Kong SMI schools without prior 
evaluation of its effectiveness, the purpose of the study is therefore to examine some aspects of 
the current practice of appraisal and provide guidelines for schools wishing to introduce teacher 
appraisal. To achieve these purposes, the following research questions are investigated: 
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1 . What are the dimensions of the outcomes of appraisal? 

2. What aspects of the appraisal system are important in determining its effectiveness? 

Method 

Subjects 

The subjects of the study were teachers of the schools which had participated in the School 
Management Initiative (SMI) Scheme. Four schools participated in a pilot study while fifteen 
schools participated in the main study. Each participating school was provided with 40 
questionnaires for distribution to teachers who had completed an appraisal cycle. For the main 
study, 360 questionnaires were returned, representing a return rate of approximately 60%. This 
return rate is quite reasonable as newly employed teachers may not have gone through a complete 
appraisal cycle at the time of survey, and were not eligible to respond to the questionnaires. 
Twenty-three questionnaires were excluded because they were incomplete, leaving 337 for 
analysis. 

The Instruments 

The questionnaire for the present study consists of two parts: one on outcomes of appraisal, 
and the other on aspects of the appraisal system. The questionnaires were pilot tested to explore 
the factor structure of the questionnaire and assess the internal consistency reliability of the 
instruments. An item was excluded if it did not load on a factor or its inclusion decreased the 
reliability of the subscale. The questionnaire was written in Chinese because some teachers who 
participated in the pilot study indicated that they were not used to questionnaires written in 
English. 

Outcomes of Appraisal 

Teachers were asked to indicate on a 5-point scale from strongly disagree (1) to strongly 
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agree (5), the extent to which they agreed with 15 statements on the outcomes of their last 
appraisal. The effectiveness of appraisal was measured by those outcomes that reflected an 
improvement in teachers' professional practices. 

Aspects of Teacher Appraisal 

On the same scale as above, teachers were asked to respond to 43 statements describing 
various aspects of their schools' teacher appraisal system as perceived by them such as attitudes 
on the purposes of appraisal, appraisal criteria, classroom observation, sources of data, 
characteristics of the appraiser, competence of the appraiser, teacher involvement, nature of 
feedback and provision of professional development in their schools. 

Data Analysis 

Exploratory principal component analysis were performed to reveal the underlying factors 
of the outcomes of appraisal and aspects of appraisal, since the study is essentially exploratory in 
nature. Oblique rotation was carried out after the factors were extracted since there was no basis 
for expecting orthogonal relationships between the dimensions of appraisal outcomes and among 
the various aspects of appraisal (Gorsuch, 1983). Correlation and stepwise multiple regression 
analyses were then performed to establish the relations between the effectiveness of appraisal and 
the aspects of the appraisal system. 



Results and Discussion 



Background Characteristics of Teachers 



Subjects who took part in the study had a mean age of 35.07 (S.D. = 8.40) and their mean 
years of teaching experience was 11.13 (S.D. = 8.31). Other background characteristics of the 
teachers are shown in Table 1. From the Table, it can be noted that approximately 60% of the 
subjects were classroom teachers who did not hold any positions of responsibility (for example, 
vice-principal, discipline, careers, or student guidance masters/mi stresses), and 70. 1% of them 
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were appraised by their subject panel chairpersons. 



Insert Table 1 about here 



Outcomes of Teacher Appraisal 

Principal components analysis of the 1 5 items of the outcomes of appraisal with oblique 
rotation revealed two factors with eigenvalues 8.45 and 1 .04 that together account for 63% of 
the total variance. Factor 1 was named formative outcomes of appraisal because it was related 
to an increase in professional practices of the teachers. Factor 2 was named summative outcomes 
of appraisal as it consisted of items about staff promotion, accountability, and managerial 
decision making. The factor solution is shown in Table 2. The two factors have high reliability 
coefficients which are .94 and .80 respectively. 



Insert Table 2 about here 



The two factors obtained are clearly about achievement of formative and summative 
purposes of appraisal as identified in the literature (Darling-Hammond, Wise and Pease, 1983; 
Popham, 1988; Stiggins, 1986). Results of the present study confirmed that there are two 
dimensions of the outcomes of appraisal - formative and summative. The formative outcomes 
factor is taken as the effectiveness of appraisal in this study. 

Aspects of Teacher Appraisal and Formative Outcomes of Appraisal 

Principal components analysis using oblique rotation revealed 10 meaningful factors with 
eigenvalue greater than 1 . These factors accounted for 64% of the total variance. The factors 
were named formative appraisal procedures, formative attitudes of appraisal, characteristics of 
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appraiser, teacher involvement in appraisal, sources of data, summative attitudes of appraisal, 
professional development, appraisal feedback, competence of appraiser and appraisal criteria. 
The factor solution is shown in Table 3. 



Insert Table 3 about here 



The means and the alpha coefficients of the subscales of the aspects of teacher appraisal are 
shown in Table 4. The subscales have moderate to high internal consistency reliability, except the 
subscales, summative attitudes of appraisal and professional development which have rather low 
alpha values. Nevertheless, they were included for further analysis because they represent 
important concepts in the appraisal system. 



Insert Table 4 about here 



Effectiveness of appraisal was found to correlate significantly and positively with the 
formative appraisal procedures (r = .58, p < .001), formative attitudes of appraisal ( r = .37, p 
< .001), characteristics of appraiser (r = .45, p < .001), teacher involvement in appraisal (r = 
.28, p < .001) , sources of data (r = .25, p < .001), professional development (r = .25, p < .001), 
appraisal feedback (r = .71, p< .001), competence of appraiser (r = .43, p < .001) and appraisal 
criteria (r = .43, p < .001). 

The relationship of effectiveness of appraisal with various aspects of teacher appraisal, taken 
together, was assessed by stepwise multiple regression analysis, and the results are shown in Table 
5. The multiple regression analysis shows that appraisal feedback, formative appraisal 
procedures and formative attitudes of appraisal are the most important predictors of the 
effectiveness of appraisal. 
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Insert Table 5 about here 



Duke and Stiggins (1990) have noted that to encourage teachers to grow professionally, it 
is important to emphasize the formative purpose of appraisal. For an appraisal system to be 
effective, it is important that the purpose be understood and perceived by teachers as helping them 
improve (Valentine, 1992) because people's behavior can be influenced by their own attitudes 
(Feldman, 1985). In this study, teachers who had formative attitudes toward the purpose of 
appraisal, that is, they perceived the purpose of appraisal is to help them improve, were also found 
to have experienced more formative outcomes. 

Stiggins and Bridgeford (1985) have noted that for formative evaluation to work most 
effectively, there should be appropriate procedures for acquiring information about teachers' 
performances. The present study suggests that an effective appraisal process is one that begins 
with a pre-classroom observation conference in which the teacher and the appraiser discuss the 
lesson content and the appraisal procedures. After the classroom observation, feedback with 
achievable goals and professional development based on appraiser’s suggestions are provided. 
Also, self-appraisal should be encouraged throughout the appraisal. 

In this study, appraisal feedback which was convincing, helpful and useful in improving 
teaching, had the strongest correlation with the effectiveness of appraisal (r=.71, p < .01). 
Multiple regression analysis showed that it was also the most important factor for predicting the 
effectiveness of appraisal and accounted for more than half the variance. Formative appraisal 
procedures and formative attitudes only accounted for an additional 3%. Weber (1987) and 
Natriello (1990b) regarded feedback as the essential element in every formative appraisal system 
because through feedback, teachers are provided with relevant information to make adjustments, 
modifications, or further development for improvement (Ovando and Harris, 1993). 
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Conclusion 

This study has found that the outcomes of appraisal can be grouped as formative and 
summative. The effectiveness of appraisal, as measured by teachers' perceptions of the formative 
outcomes, was predicted by teachers' formative attitudes towards appraisal, the appraisal 
procedures, and feedback. Teachers are more likely to benefit from appraisal if they perceive the 
appraisal as helping them grow professionally, the appraisal procedures are consistent with 
formative purposes, and the feedback provided is useful. 
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Characteristics Number Percentage 



Sex 



JCa 

Male 


161 


48.5 


Female 


171 


51.5 


Academic Qualification 
Tertiary Non-degree 


95 


28.4 


Bachelor 


201 


60.0 


Master 


33 


9.8 


Others 


6 


1.8 


Professional Qualification 
Teachers' training 


279 


83.3 


Nil 


56 


16.7 


Position at School 

Teacher (with no positions of responsibility) 


204 


60.5 


Subject Panel Chairperson 


96 


28.5 


Administration (e g. Vice-principal, discipline. 


37 


11.0 


careers, or guidance master/mi stress) 

Appraiser 

Teachers 

Panel Chairperson 


143 


70.1 


Vice-Principal 


19 


9.3 


Principal 


42 


20.6 


Panel Chairperson 
Vice-Principal 


45 


47.4 


Principal 


50 


52.6 


Administration 

Panel Chairperson 


2 


5.6 


Vice-Principal 


8 


22.2 


Principal 


26 


72.2 



Note. Total n = 337, missing data deleted listwise. 
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Table 2 

Principal Components Analysis of the Outcomes of Teacher Appraisal Items 



Number 


Item 


I 


II 




Factor I: Formative Outcomes of Appraisal 






3 


Care more about teaching 


.95 


-.21 


2 


Reflect more on teaching 


.92 


-.15 


11 


Understand strengths and weaknesses better 


.82 


.03 


1 


Increase in teaching skill 


.78 


-.02 


8 


Understand teaching-learning process better 


.76 


.13 


12 


Consider teaching more like a professional job 


.71 


.16 


14 


Know the direction of professional development 


.71 


.09 


15 


Gain more reinforcement in teaching 


.63 


.16 


5 


Provide professional development needs of teachers 


.59 


.21 


4 


Know the areas that teachers require improvement 


.58 


.22 


13 


Help teachers improve teaching effectiveness 


.60 


.34 




Factor II: Summative Outcomes of Appraisal 






7 


Fair staff promotion 


-.03 


.82 


6 


Fair assessment of teacher performance 


.05 


.77 


10 


Increase accountability of school 


.05 


.71 


11 


Managerial decision making 


.26 


.61 
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Table 4 

Means, Standard Deviations and Reliabilities of the Subscales of 
Aspect of Appraisal 



Subscale 


Mean 


S.D. 


Alpha coefficient 


Formative appraisal procedures 


2.92 


0.63 


.75 


Formative attitudes of appraisal 


3.61 


0.73 


.73 


Characteristics of appraiser 


3.41 


0.68 


.70 


Teacher involvement in appraisal 


2.71 


0.96 


.68 


Sources of data 


3.29 


0.83 


.66 


Summative attitudes of appraisal 


3.05 


0.83 


.41 


Professional development 


3.09 


0.70 


.49 


Appraisal feedback 


3.00 


0.74 


.87 


Competence of appraiser 


3.24 


0.66 


.66 


Appraisal criteria 


3.24 


0.65 


.60 



Table 5 

Stepwise Multiple Regression Analysis for the Relationship between 
Effectiveness of Appraisal and Aspects of Teacher Appraisal 



Step 


Variable 


P 


R 2 


Change 
in R 2 


F 


1 


Appraisal feedback 


.56 


.51 


.51 


302.97*** 


2 


Formative appraisal procedures 


.17 


.53 


.02 


164.52*** 


3 


Formative attitudes of appraisal 


.12 


.54 


.01 


1 14.93*** 



Note. ***p< .001. 
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